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1. INTRODUCTION 

Pose estimation is a desired position and orientation of the inspected object before inspection or 
before robot manipulation task will take place [1]. Technological advancement on product design modelling 
influenced on product shape complexity [2]. Shape complexity such as free form line, almost symmetrical 
shape, and product features such as hole, fillet, chamfer brings difficulty on product pose estimation and 
inspection. This task usually done by dedicated tools, fixtures, and other part presentation devices [3]. 
The problem with this approach is that it takes undesirable amounts of debug and support time and is 
sometimes very expensive. By knowing the object pose before inspection will contributes on less time 
consuming for overall object inspection. External surface of whole object inspection is an essential 
technology operation in the production line of diverse objects [3, 4]. 

Although there are many methods proposed for evaluation of partial pose estimation, most of them 
are sensitive to noise such as shadow and insufficient object internal information especially for almost 
symmetrical part [2]. To address the problems, we propose a robust global shape representation using 
threshold binary region shape description with outer box interest point method. This outer box interest points 
contributes to object rescaling alignment and image registration. 

Previous study presented various techniques on CAD image database development [5] using .stl 
data, image data acquisition for surface inspection [6], pre-processing techniques on image registration 
[7-18], features extraction such as Vote-based 3D shape recognition and registration [19], edges extraction 
[20], reflection symmetric [21], DoG-based detector presented by deriving scale-invariant mesh features for 
image registration [22], Local Procustes Regression (LPR) [23], Estimation-by-Completion (EbC) [24] and 
Customized three-dimensional template matching [25-27] has been conducted. However, issues in pose 
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estimation was still an open problem in partial pose estimation especially for almost symmetrical shape 
of object. 

In this paper, the study focus on object description, scaling and image registration method. 
The contribution of this paper two fold. First, effective scale method to align two or more source image using 
hypotenuse length of the outer box ratio. In this work, object scale ratio needed to overcome different size 
object in inspected image and CAD model image. This is important because of natural phenomena of object 
size where are the CCD camera placement distance influence the object size and CAD model object size 
influenced by the image zoom in and zoom out. Second, segmented image was using interest point of object 
outer box method and produced different size image. Different size segmented image was align using unique 
image padding used in image registration stage. 

The rest of this paper is organized in four sections. Section | contains introduction to partial pose 
estimation and its related studies. Section 2 introduces methodology of the proposed partial pose estimation 
algorithm using outer box method. This section was divided into four steps of rescaling, image registration, 
correlation matching and robustness test. The Outer Box method experimental results was presented in 
Section 3. Comparisons with existing method was carried out in Section 3.3 and Section 4.0 summarizes the 
conclusions of this work. 


2. RESEARCH METHODOLOGY 

Development on partial pose estimation using CAD database algorithm for non contact inspection in 
industry was carried out. Its consists of three main contributions in processing steps which were alignment of 
object size scale in image, segmented image padding method in image registration stage and lastly image 
retrieval process. 


2.1. Partial Pose Estimation Of Tested Object To CAD Database 

Figure 1 shows flowchart for partial pose estimation development using CAD database. As shown in 
this figure, acquisited CAD image and tested image go through rescaling and image registration stage. 
The inspected image will retrieve the similar pose in CAD model database. Extracted object using Outer Box 
method was used in image segmentation before image registration stage takes place. Both sources of tested 
image and CAD image go through this process. Collection of CAD model images in database act as reference 
poses. CAD model poses for 1 degree rotation from O degree to 360 degree produced 360 images for +1 
degree accuracy of retrieve pose. 
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Figure 1. Flowchart representation for partial pose estimation development 
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2.1.1 Step 1: Outer Box For Object Size Rescaling 

Hypotenuse of outer box method is a new approach for object rescaling introduced in this 
experiment. This method using combination of scale and rotation invariance with the local nature of object 
pose features. As shown in (1) and (2) were used to get salient point on outer box. 
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where li is intensities of pixels I, I; is intensities of pixels j. As shown in (1) used to get coordinate of 
minimum point and (2) to get coordinate of maximum point of outer box. By determine that points, 
hypotenuse length was obtained using (3). Using this value, ratio of rescaling image size on CAD object and 
PRI object was calculated and applied for image processing stage. 
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Where B = coordinate of point B, (Bimax, Bjmax),, A = coordinate location of point A, (Aimin , Ajmin) and c 
= hypotenuse length of outer box method. 


2.1.2 Step 2: Image Padding For Image Registration 

After rescaling process, image padding in image registration process was takes place. 
Image registration is a stage to aligning image size for inspected image and CAD image. Image padding was 
using (4). 


Tnew = ([lo Io Io; To I Io; 10 Io Io) (4) 


Where Inew = the segmented image, Io = the zero padding image same size to I image and I = the extracted 
object image using outer box image method. 


2.1.3 Step 3: Feature Extraction and Correlation Matching Retrieval 

Template matching method is a technique to find similar recognize object feature in inspected image 
to reference image in CAD database. The basic template-matching algorithm involves sliding the template 
image over the source image and at each position calculating a 2D correlation of the binary image using pixel 
binary difference to estimate the degree of similarity between the template (CAD image) and source image 
(inspected image). 2D correlation in (5) was used in template-correlation matching algorithms. 
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Where A is CAD image, B is PRI image, A is mean2(A), B is mean2(B). 

Figure 2 show the tested object for this partial pose estimation experiment. This figure contained, (a) 
automotive component (b) bottom of automotive component (c) automotive component with label (d) 
Arduino board (e) computer mouse and (f) USB connector. Ten test pose estimation was implemented for 
each object. 
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(a) (c) 


Figure 2. Simulation on (a) automotive component (b) bottom of automotive component 
(c) automotive component with label (d) Arduino board (e) computer mouse and (f) USB connector 





(d) (e) (f) 


2.2. Robustness Test 
2.2.1 Step 1: Random Database 

To analyse the robustness of the develop algorithm, random database structure arrangement was 
created in unique names. This test was performed to observe image retrieval in random CAD model database. 
Figure 3(a) show the image naming in CAD model file database. This test was implemented for automotive 
cast part in +1 degree accuracy. 


2.2.2 Step 2: Mixed Model Database 

Another test was carried out in order to observe the robustness of this algorithm by mixing CAD 
models in CAD file database. Three trials for each object of automotive component, bottom of automotive 
component, Arduino board, computer mouse and USB connector were carried out. Figure 3(b) shows the 
CAD model image in file database. 
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Figure 3. (a) CAD model images with unique name in file database and (b) Mixed CAD model images 
in file database 


2.3. Accuracy Test for Partial Pose Estimation 

To verify the accuracy, three reference points of Outer Box triangulation properties of test object 
and CAD model was calculated as in (6). The comparison angle value determined the accuracy of the 
propose method. 
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Where 0 is angle error, Bimax is pixel i coordinate at point maximum of outer box and Ajmin is pixel i 
coordinate at point minimum outer box. 


3. EXPERIMENTAL RESULTS AND ANALYSIS 

Figure 4 show hardware set up for inspected object image acquisition. Single CCD camera was 
placed at top view during image acquisition. CAD model database development using SolidWorks platform 
and saved into .slp file format. Using this .slp model, the CAD model rotated at z axis for image acquisition 
at 0 interval. The 0 was set at 1 degree rotation during image acquisition and results on structure arrangement 
of CAD model database. 


Partial pose estimation of 3D rigid object system using outer box method (Z. Zulkoffli) 


794 g ISSN: 2502-4752 





Figure 4. Hardware set up for inspected object image acquisition 


3.1. Pre-Processing (Rescaling, Image Padding and Image Segmentation) 

The partial pose estimation measurement system works as follows. First, the test image and CAD 
image go through binary image filtering at certain threshold value, this is purpose to extract the object from 
the background. For example, automotive component set at >30 threshold value on test image and set at >50 
threshold value on CAD image. After object extraction, the outer box method applied on both object. 
Interest points on Outer box gives information on hypotenuse length on the test object and CAD object. 
Using this hypotenuse length, standard object size and CAD object size determined. However, using this 
outer box method, hypotenuse length varied at all different poses. In this stage, hypotenuse length ratio to 
CAD object and tested object was use to resize the image at standard scale. After rescaling, image size varied 
for both test image and CAD image. The extracted image padded with same size zero pixel image around the 
extracted image using (4). 

Figure 5 shows the rescaling object, image padding and segmented image results for both inspected 
object image and CAD object image for single pose of tested objects. The image padding for inspected image 
was varied based on size of the inspected object outer box. The image padding for CAD image was varied on 
size of CAD model object outer box. Standardize image size through image registration performance was 
produce at a standard image size for both inspected image and CAD image. 
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Figure 5. Extracted information on simulation object 


3.2. Processing (Image Retrieval Using Correlation Matching) 

Image retrieval using correlation matching of tested image to all CAD images in database. 
Highest correlation value determined the similar pose estimation of the tested image. Figure 6 shows the 
tested image and its retrieved CAD model pose in database for all tested objects. The tested object results on 
set of tested object together with retrieved CAD model and image retrieval graph for pose in CAD database. 
The graph marked with the highest peak that indicate the similar pose of tested object and CAD model pose. 
Table 1 indicate the average recognition rate, average processing time, average accuracy for all tested object 
within ten times trial at different pose. In this table, average recognition rate was varied for different object 
model. Various recognition rate value were caused by shape representation information on the tested object 
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and CAD model such as corner and edge. Recognition rate value were calculated by 2D correlation matching 
similarity of tested object image and CAD model image, this fully beneficial in recognizing significant object 
area with internal shape information. Average processing time also different for different tested object model 
and depending on information on the image. Average accuracy obtained also showed different average 
accuracy on different test model. However, in overall, the accuracy fall in range +0.83° to +1.14° value. 
Planar shape component such as Bottom Automotive Component shows the highest accuracy obtained at 
+0.83". Little shadow appearance on computer mouse slightly interrupt the image retrieval process at 81.86% 
recognition rate within +1.05° accuracy. Different manufacturing material of USB connector shows some 
part of the tested object did not fully transform into binaries during image retrieval process, in this case the 
different of metal and plastic part did not have same value threshold. Threshold value for plastic was >30, 
while for metal was <5. Although partial of object information provided, this method successfully identified 
partial pose estimation within +1.13° accuracy. 
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Figure 6. Correlation matching results for automotive component, bottom automotive component, 
Arduino board, computer mouse and USB connector 


Table 1. Recognition Rate and Image Retrieval Performance of Different Type Object 


Object Average recognition rate Average processing time (s) Average accuracy 
Automotive component 93.78 % 1009.4s +1.14° 
Bottom automotive component 76.24% 986.0s +0.83° 
Arduino Board 84.86% 938.2s +1.03° 
Computer mouse 81.86% 1034.0s +1.05° 
USB connector 84.54% 1141.8s +1.13? 


3.3. Robustness 
3.3.1 Tested Object With Labeled Noise 

Figure 7 shows tested image retrieval for automotive component with label to CAD model database. 
Ten tests were done in this condition and achieved average retrieval accuracy at +1.14° in this experiment. 
Table 2 shows the average recognition rate, processing time and average accuracy for this experiment. 
Label appearance on the tested object does not affect the effectiveness of image retrieval process, but slightly 
influence lower correlation value down to 7.33% on retrieved pose. Processing time taken for labelled 
automotive component shows increases up to 19.9s compared to non label automotive component. In Table 
2, column two shows single pose image retrieval for automotive component to random structure CAD model 
database. Ten test were done in this condition and resulted on average accuracy at +1.15° in this experiment. 
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Average recognition rate for random CAD database shows no significant difference compared to structure 
CAD database within 0.03% and no difference on average processing time. The results shows similar 
correlation value of pose estimation retrieved in this process compared to structure CAD model database. 
Random CAD model database influence all correlation graph structure and not effect on retrieval value 
as in Table 2. 
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Figure 7. Image retrieval using correlation matching of automotive component at (a) labelled noise and 
(b) within random CAD database 


Table 2. Recognition Rate and Image Retrieval Performance within Noise Information 


Condition Average Recognition rate Processing time (s) Average Accuracy 
Labelled automotive component 86.45% 1029.3s +1.14° 
Random arrangement database 93.75% 1009.4s bl IS 


3.3.2 Image Retrieval in Mixed CAD Database 

Another robustness test was implemented using image retrieval in mixed CAD model database had 
been carried out. Three different poses were tested in this mixed CAD model database for automotive 
component, bottom automotive component, Arduino board, computer mouse and USB connector. Table 3 
show average accuracy at +1.14° for automotive component, +0.38° for bottom automotive component, 
+1.05° for Arduino board, +1.07° for computer mouse and +1.12° for USB connector. Average recognition 
rate also shows different range value for different model, which the highest for automotive component at 
90.81% and the lowest for bottom automotive component at 73.34%. Average processing time varied for 
different model in range 464.8s to 572.9s. This processing time taken in 180 images CAD database. 
This proved, this feature representation is robust in mixed CAD model database. 


Table 3. Recognition Rate and Image Retrieval Performance in Mixed CAD Model Database 


Mixed model database Average Recognition rate Average Processing time (s) Average Accuracy 
Automotive component 90.81% 477.88 ool 
Bottom automotive component 73.34% 551.8s +0.38° 
Arduino Board 81.57% 464.8s +1.05° 
Computer mouse 88.32% 516.7s +1.07° 
USB connector 75.55% 572.9s +1.12° 


3.4. Comparisons With Existing Method 

Table 4 shows comparison accuracy result of the proposed method and previous study. In this table, 
this presented work shows a highly competitive results on accuracy in this partial pose estimation 
identification of tested object to CAD model database within range +0.388° to +1.471° error. Naoki et. al 
[28], Carlos et. al [8] and Toshiyuki & Toru [26] development obtained pose estimation error at +5.81, +1.50 
to +4.84 and +2.26 to +2.07 degrees, respectively. 
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Table 4. Comparison of Propose Method to Previous Study 


Method Accuracy (degree) 
Outer Box Correlation Matching +0.38° to +1.47° 
Naoki et. al [28] +5.81° 
Carlos et. al [8] +1.50° to +4.84° 
Toshiyuki & Toru [26] +2.26 to + 2.07 


4. CONCLUSIONS 

In this paper, a novel approach for partial pose estimation identification was introduced that using 
geometric information from CAD models. This work was successfully able to combine the appearance 
information of tested object to CAD models image. Notably, our algorithm outperforms on accuracy and 
introduced feature robustness, and significantly improves existed method on partial pose estimation 
identification. We believe that our work provides a platform to tackle higher level tasks such as 3D pose 
estimation identification and multi view image registration. 
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